feat: Global Inference for Bedrock models (#8750) #8940

hannesrudolph · 2025-10-30T22:40:39Z

Related GitHub Issue

Closes: #8750

Roo Code Task Context (Optional)

No Roo Code task context for this PR

Description

Implements Global Inference support for Amazon Bedrock models that currently support it (Claude Sonnet 4, Claude Sonnet 4.5, Claude Haiku 4.5). Adds:

New constant BEDROCK_GLOBAL_INFERENCE_MODEL_IDS to declare supported models.
New settings flag awsUseGlobalInference (types and UI).
Provider logic in AwsBedrockHandler to prefix global. for supported models (takes precedence over cross‑region); parseBaseModelId strips the prefix.
UI toggle under Bedrock provider, shown only for supported models. Enabling Global Inference disables cross‑region.

Test Procedure

Unit tests:
- cd src && npx vitest run api/providers/tests/bedrock.spec.ts api/providers/tests/bedrock-inference-profiles.spec.ts
- cd webview-ui && npx vitest run
Manual:
- In Settings → Providers → Bedrock, select a supported model (Sonnet 4 / 4.5 / Haiku 4.5).
- Toggle “Use Global inference”. Cross‑region is disabled automatically.
- Verify requests use global. and that parsing removes the prefix for model metadata.

Pre-Submission Checklist

Issue Linked
Scope limited to feature
Self-review completed
Tests pass locally
Documentation/i18n considered (English string added; other locales can be updated separately)
No breaking changes

Screenshots / Videos

No UI layout changes; adds one checkbox when applicable.

Documentation Updates

No documentation updates are required.

Additional Notes

Global Inference toggle only appears for models in BEDROCK_GLOBAL_INFERENCE_MODEL_IDS.

Get in Touch

@your-discord

Important

Adds global inference support for specific Amazon Bedrock models, updating settings, UI, and tests to accommodate the new feature.

Behavior:
- Adds global inference support for Amazon Bedrock models (Claude Sonnet 4, Claude Sonnet 4.5, Claude Haiku 4.5) in bedrock.ts.
- Introduces awsUseGlobalInference setting in provider-settings.ts and UI toggle in Bedrock.tsx.
- AwsBedrockHandler prefixes model IDs with global. for supported models when global inference is enabled.
Constants:
- Adds BEDROCK_GLOBAL_INFERENCE_MODEL_IDS in bedrock.ts to list supported models.
Tests:
- Updates tests in bedrock-inference-profiles.spec.ts and bedrock.spec.ts to cover new global inference logic.
Translations:
- Adds awsGlobalInference translation key in multiple locale files (e.g., settings.json for ru, tr, vi, zh-CN, zh-TW).

^{This description was created by}^{for 598ccac. You can customize this summary. It will automatically update as commits are pushed.}

roomote · 2025-10-30T22:41:00Z

Code Review Summary

I've reviewed the latest changes and found no new issues. The addition of ap-southeast-4 (Melbourne) to the Australia regional inference profile is correctly implemented.

Issues Found

Missing mutual exclusion in UI: The cross-region inference checkbox should also disable global inference when enabled, creating proper mutual exclusion between the two settings. Currently only one direction is handled.
Translation error in Hindi locale: The Hindi translation contains Gujarati script. The word "શ્રેષ્ઠ" should be "श्रेष्ठ" in Hindi (Devanagari script).

Previous Reviews

_{Mention @roomote in a comment to trigger your PR Fixer agent and make changes to this pull request.}

webview-ui/src/components/settings/providers/Bedrock.tsx

Copilot

Pull Request Overview

This PR adds support for AWS Bedrock Global Inference profile routing, which enables automatic selection of optimal AWS regions for supported models. The feature is mutually exclusive with cross-region inference.

Key changes:

Added new configuration option awsUseGlobalInference for enabling Global Inference profiles
Introduced BEDROCK_GLOBAL_INFERENCE_MODEL_IDS constant listing supported models (Claude Sonnet 4, 4.5, and Haiku 4.5)
Implemented UI checkbox that appears conditionally for supported models and enforces mutual exclusivity with cross-region inference

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
webview-ui/src/i18n/locales/en/settings.json	Added translation key for Global Inference checkbox label
webview-ui/src/components/settings/providers/Bedrock.tsx	Added conditional UI checkbox for Global Inference with one-way mutual exclusion logic
src/api/providers/bedrock.ts	Implemented Global Inference prefix (`global.`) application logic with precedence over cross-region inference
packages/types/src/providers/bedrock.ts	Defined list of models supporting Global Inference
packages/types/src/provider-settings.ts	Added `awsUseGlobalInference` boolean field to Bedrock schema

Comments suppressed due to low confidence (1)

webview-ui/src/components/settings/providers/Bedrock.tsx:167

The cross-region inference checkbox lacks reciprocal mutual exclusion logic. When enabling cross-region inference, it should also disable Global Inference (similar to how enabling Global Inference disables cross-region inference on lines 157-158). Update the onChange handler to: onChange={(checked: boolean) => { setApiConfigurationField('awsUseCrossRegionInference', checked); if (checked) setApiConfigurationField('awsUseGlobalInference', false); }}

			<Checkbox
				checked={apiConfiguration?.awsUseCrossRegionInference || false}
				onChange={handleInputChange("awsUseCrossRegionInference", noTransform)}>
				{t("settings:providers.awsCrossRegion")}
			</Checkbox>

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…erence (disable global inference)

…anguages

webview-ui/src/i18n/locales/hi/settings.json

… enabled

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

roomote

The mutual exclusion issue has been resolved. However, there is still one open issue regarding the Hindi translation that needs to be addressed before approval.

XL-Lewis · 2025-11-04T03:21:02Z

@roomote the jp and au (and potentially others) prefixes are currently missing from AWS_INFERENCE_PROFILE_MAPPING which need to be added as well: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html

roomote · 2025-11-04T03:23:21Z

Fixed the reported issues. All local checks passed.

_{View commit | Revert commit}

XL-Lewis · 2025-11-04T03:28:14Z

@hannesrudolph Apologies - missed that this was a bug due to the addition of new, more specific regions by AWS.

…an and Australia - Add jp. prefix for ap-northeast- regions (Tokyo/Osaka) - Add au. prefix for ap-southeast-2 region (Sydney) - Reorder mappings by pattern length descending for correct matching - Update tests to reflect new regional mappings - Allow numbers in region pattern validation regex Addresses feedback from XL-Lewis per AWS documentation: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html

XL-Lewis · 2025-11-04T03:36:15Z

@roomote The au region covers both ap-southeast-2 and ap-southeast-4, while jp covers 1 and 3

roomote · 2025-11-04T03:38:25Z

Fixed the reported issue. All local checks passed.

_{View commit | Revert commit}

…e profile

feat: add Global Inference for Bedrock models (#8750)

a02975d

Copilot AI review requested due to automatic review settings October 30, 2025 22:40

hannesrudolph requested review from cte, jr and mrubens as code owners October 30, 2025 22:40

github-project-automation bot added this to Roo Code Roadmap and Roo Code Roadmap Oct 30, 2025

github-project-automation bot moved this to Triage in Roo Code Roadmap Oct 30, 2025

github-project-automation bot moved this to New in Roo Code Roadmap Oct 30, 2025

hannesrudolph mentioned this pull request Oct 30, 2025

[ENHANCEMENT] Global Inference for Bedrock models supported #8750

Open

2 tasks

dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. UI/UX UI/UX related or focused labels Oct 30, 2025

roomote bot reviewed Oct 30, 2025

View reviewed changes

webview-ui/src/components/settings/providers/Bedrock.tsx Outdated Show resolved Hide resolved

Copilot AI reviewed Oct 30, 2025

View reviewed changes

hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 30, 2025

fix(bedrock): enforce mutual exclusion when enabling cross-region inf…

b4f288c

…erence (disable global inference)

roomote bot approved these changes Oct 30, 2025

View reviewed changes

feat(i18n): add global inference option to AWS settings in multiple l…

15526fa

…anguages

roomote bot reviewed Oct 30, 2025

View reviewed changes

webview-ui/src/i18n/locales/hi/settings.json Outdated Show resolved Hide resolved

ellipsis-dev bot reviewed Oct 30, 2025

View reviewed changes

webview-ui/src/i18n/locales/hi/settings.json Outdated Show resolved Hide resolved

hannesrudolph and others added 2 commits October 30, 2025 17:48

fix(bedrock): disable cross-region inference when global inference is…

f87e1b9

… enabled

Update webview-ui/src/i18n/locales/hi/settings.json

4493873

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

hannesrudolph moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Oct 30, 2025

roomote bot reviewed Oct 30, 2025

View reviewed changes

roomote bot approved these changes Oct 30, 2025

View reviewed changes

hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Oct 31, 2025

fix(bedrock): add ap-southeast-4 (Melbourne) to au. regional inferenc…

598ccac

…e profile

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Global Inference for Bedrock models (#8750) #8940

feat: Global Inference for Bedrock models (#8750) #8940

Uh oh!

hannesrudolph commented Oct 30, 2025 •

edited by ellipsis-dev bot

Loading

Uh oh!

roomote bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

roomote bot left a comment

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

roomote bot commented Nov 4, 2025 •

edited

Loading

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

roomote bot commented Nov 4, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: Global Inference for Bedrock models (#8750) #8940

Are you sure you want to change the base?

feat: Global Inference for Bedrock models (#8750) #8940

Uh oh!

Conversation

hannesrudolph commented Oct 30, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related GitHub Issue

Roo Code Task Context (Optional)

Description

Test Procedure

Pre-Submission Checklist

Screenshots / Videos

Documentation Updates

Additional Notes

Get in Touch

Uh oh!

roomote bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Summary

Issues Found

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

roomote bot left a comment

Choose a reason for hiding this comment

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

roomote bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

XL-Lewis commented Nov 4, 2025

Uh oh!

roomote bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hannesrudolph commented Oct 30, 2025 •

edited by ellipsis-dev bot

Loading

roomote bot commented Oct 30, 2025 •

edited

Loading

roomote bot commented Nov 4, 2025 •

edited

Loading

roomote bot commented Nov 4, 2025 •

edited

Loading